Cameras are commonly used to capture an image of a scene that includes one or more objects. Unfortunately, some of the images can be blurred. For example, movement of the camera and/or movement of the objects in the scene during the exposure time of the camera can cause the image to be blurred. Further, if the camera is not properly focused when the image is captured, the image can be blurred.
Deconvolution methods used for deblurring tend to be very complex and slow (with the exception of the most simple methods that produce very low quality images) and deblurred images tend to contain a lot of strong artifacts, most often in the form of ringing around edges in the image. Also, there is a tendency to strongly magnify noise. Techniques that help to reduce ringing artifacts and noise magnification in deconvolved images however tend to reduce any fine texture. Thus, the resulting images have an unnatural, posterized look, as many areas in these images appear unnaturally smooth and the textures are blurry and muddled.
When blur is sufficiently spatially uniform, a blurred captured image can be modeled as the convolution of a latent sharp image with some point spread function (“PSF”) plus noise,B=K*L+N.  Equation(1).In Equation 1 and elsewhere in this document, (i) “B” represents a blurry image, (ii) “L” represents a latent sharp image, (iii) “K” represents a PSF kernel, and (iv) “N” represents noise (including quantization errors, compression artifacts, etc.).
A non-blind deconvolution problem seeks to recover the latent sharp image L only when the PSF kernel K is known. Alternatively, a blind deconvolution problem seeks to recover both the PSF kernel K and the latent sharp image L. Both the blind deconvolution problem and non-blind deconvolution problem are difficult to accurately solve because they are ill conditioned, and some information has been irretrievably lost due to zero and near zero values in the corresponding optical transfer function (“OTF”), Fourier transform of the PSF.
Moreover, many blurry images include areas that further complicate the problem of recovering a latent sharp image L. For example, extremely bright areas where the sensor pixels reach their saturation point in the blurry image B can adversely influence the determined PSF kernel K and the resulting latent sharp image L.
One common approach to solving a deconvolution problem includes reformulating it as an optimization problem in which suitable cost functions are minimized in order to find a solution to the deconvolution problem.
A relatively common type of a cost function used for deconvolution is a regularized least squares cost function. Typically, a regularized least squares cost function consists of (i) one or more fidelity terms, which make the minimum conform to equation (1) modeling of the blurring process, and (ii) one or more regularization terms, which make the solution more stable and help to enforce prior information about the solution, such as sparseness.
An example of a generalization of a latent sharp image cost function isc(L)=∥K*L−B∥pp+ω∥(Dx*L)∥qq+∥(Dy*L)∥qq).  Equation(2)In Equation (2), and in all other formulas in this document (i) the subscript p denotes the norm for the fidelity term, (ii) the superscript p denotes the power for the fidelity term, (iii) the subscript q denotes the norm for the regularization terms, (iv) the superscript q denotes the power for the regularization terms, and (v) Dx and Dy denote the convolution kernels used for computing derivatives in x- and y-direction, respectively. Equation 2 is a regularized least squares cost function when p is equal to two. The first term in Equation (2), ∥K*L−B∥pp, is the fidelity term that forces the solution to satisfy blurring model in Equation (1) with a noise term that is small. Further, in Equation (2), ∥(Dx*L)∥qq and ∥(Dy*L)∥qq are regularization terms that help to infuse prior information about arrays that can be considered a latent sharp image. The derivatives of a natural image tend to be sparse: in most points derivatives are close to zero and only in relatively few points (edge points) the values are large. The regularization terms help to enforce this property. In Equation (2), ω is a regularization weight, a selected constant that helps to achieve the proper balance between the fidelity term and the regularization terms. It should be noted that this example represents a very simple cost function. Often, more complicated cost functions are used, such as those with more fidelity and regularization terms involving derivatives of 1st or higher order.
When the image derivatives are assumed to follow a Guassian prior distribution, the power (superscript) and norm (subscript) for the regularization term(s) in the cost function is equal to two (q=2) (also referred to as a “Gaussian prior” or “2-norm”), and the cost function is often referred to as a Gaussian prior cost function. Unfortunately, the resulting latent sharp image from a Gaussian prior cost function often include strong ringing artifacts in the vicinity of high contrast edges.
In an attempt to improve the resulting latent sharp image, other powers and norms have been tried in the regularization term(s). For example, when image derivatives are assumed to follow a Laplacian prior distribution, the resulting regularization term in the cost function has a power and a norm that are equal to one (q=1) (also referred to as a “Laplacian prior” or a “1-norm”); and when image derivatives are assumed to follow a hyper-Laplacian prior distribution, the resulting regularization term in the cost function has a power and a norm that are equal to less than one (q<1) (also referred to as a “hyper-Laplacian prior” or “q-pseudonorm”). With a Laplacian or hyper-Laplacian prior cost function, the ringing artifacts are reduced, but the fine texture in the image is often not recovered and the image does not look natural.